Convergence and Implicit Regularization Properties of Gradient Descent for Deep Residual Networks

We prove linear convergence of gradient descent to a global minimum for the training deep residual networks with constant layer width and smooth activation function. further show that trained weights, as function index, admits scaling limit which is H\"older continuous depth network tends infinity. The proofs are based on non-asymptotic estimates loss norms weights along path. illustrate relevance our theoretical results practical settings using detailed numerical experiments supervised learning problems.